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any user. 

[0007] In order to achieve high accuracy speech recognition it is important that a voice controlled device avoid 
responding to speech that isn't directed to It. That iSj voice controlled devices should not respond to background con- 
versation, to noises, or to commands to other voice controlled devices. However, filtering out background sounds must 

5 not be so effective that it also prevents recognition of speech directed to the voice controlled device. Finding the right 
mix of rejection of background sounds and recognition of speech directed to a voice controlled device Is particularly 
challenging in speaker-independent systems. In speaker-independent systems, the voice controlled device must be 
able to respond to a wide range of voices, and therefore can not use a highly restrictive filter for background sounds. In 
contrast, a speaker-dependant system need only listen for a particular person's voice, and thus can employ a more 

10 stringent filter for background sounds. Despite this advantage in speaker dependant systems, filtering out background 
sounds is still a significant challenge. 

[0008] In some prior art systems, background conversation has been filtered out by having a user physically press 
a button in orderto activate speech recognition. The disadvantage of this approach is that it requires the user to interact 
with the voce controlled devfee physically, rather than strictly by voice or speech. One of the potential advantages of 

15 voce controlled devices is that they offer the promise of true hands-free operation. Elimination of the need to press a 
button to activate speech recognition would go a long way to making this h^ds-free objective achievable. 
[0009] Additionally, in locations with a number of people talking, a voice controlled device should disregard all 
speech unless it is directed to it For example, If a person says to another person "I'll call John", the cellphone in his 
pocket should not Interpret the "call John" as a command. If there are multiple voice controlled devices In one location, 

20 there should be a way to uniquely identify which voice controlled device a user wishes to control. For example, consider 
a room that may have multiple voice controlled telephones - perhaps a couple of desktop phones, and multiple cell- 
phones - one for each person. If someone were to say "Call 555-1212", each phone may try to place the call unless 
there was a means for them to disregard certain commands. In the case where a voice controlled device is to be con- 
trolled by multiple users, It Is desirable for the voice controlled device; to know which user is commanding It. For exam- 

25 pie, a votee controlled desktop phone In a house may be used by a husband, wife and child. Each would could have 
their own phonebook of frequently called numbers. When the voice controlled device is told "Call Mother", it needs to 
know wh'ich user is issuing the command so that it can call the right person (I.e. should It call the husbands mother, the 
wife's mother, or the child's mother at her work number?). Additionally, a voice controlled device with multiple users may 
need a method to enforce security to protect it from unauthorized use or to protect a user's personalized settings from 

30 unintentional or maiteious interactions by others (including snooping, changing, deleting, or adding to the settings). Fur- 
themlore, in a location where there are multiple voice controlled devices, there should be a wey to identify the presence 
of vok:e controlled devkses. For example, consider a traveler arriving at a new hotel room. Upon entering.the hotel room, 
the traveler would like to know what voice controlled devices may be present and how to control them. It Is desirable 
that the Identification process be standardized so that all voice controlled devices may be Identified In the same way. 

35 [001 0] In votee controlled devices. It is desirable to store phrases under voice control. A phrase is defined as a sin- 
gle word, or a group of words treated as a unit. This storing might be to set options or create personalized settings. For 
example, in a voice-controlled telephone, it is desirable to store people's names and phone numbers under voice control 
Into a personalized phone book. At a later time, this phone book can be used to call people by speaking their name (e.g. 
"Cellphone callJohn Smith", or "Cellphone call Mother"). 

40 [0011] Prior art approaches to storing the phrase ("John Smith") operate by storing the phrase in a compressed, 
uncompressed, or transformed manner that attempts to preserve the actual sound. Detection of the phrase in a com- 
mand (I.e. detecting that John is to be called in the example above) then relies on a sound-based comparison between 
. the original stored speech sound and the spoken command. Sometimes the stored waveform is transformed into the 
frequency domain and / or is time adjusted to facilitate the match, but in any case the fundamental operation being per- 

45 formed is one that compares the actual sounds. The stored sound representation and comparison 1br detection suffers 
from a number of disadvantages. If a speaker's voice changes, pertiaps due to a cold, stress, fatigue, noisy or distorting 
connection by telephone, or other factors, the comparison typically Is not successful and stored phrases are not recog- 
nized. Because the phrase is stored as a sound representation, there Is no way to extract a text-based representation 
of the phrase. Additionally, storing a sound representation results In a speaker dependent system. It is unlikely that 

50 another person could speak the same phrase using the same'sounds in a command and have it be correctly recog- 
nized. If would not be reliable, for example, for a secretary to store phonebook entries and a manager to make calls 
using those entries. It Is desirable to provide a speaker independent storage means. Additionally, if the phrases are 
stored as sound representations, the stored phrases can not be used In another voice controlled device unless the 
same wavefonn processing algorithms are used by both voice controlled devices. It Is desirable to recognize spoken 

55 phrases and store them In a representation such that, once stored, the phrases can be used for speaker Independent 
recognition and can be used by multiple voice controlled devtees. 

[0012] Presently computers and other devices communicate commands and data to other computers or devices 
using modern, infrared or wireless radio frequency transmission. The transmitted command and/or data are usually of 
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FlGs. 10A-10C are flow charts of the "GETRESPONSE" function for the standard voice user Interface of the 
present Invention. 

FIG. 1 1 is a . flow chart of the 'GETRESPONSEPLUS" function for the standard voice user interface of the present 
5 invention. 

FIG. 1 2 is a flow chart of the "LISTANDSELECT function for the standard voice user interface of the present inven- 
tion. 

10 FIG. 13 is a block diagram of a pair of voice controlled devices communicating using the standard voice user inter- 
face of the present invention. 

like reference numbers and designations in the drawings indicate like elements providing similar functionality. 
DETAILED DESCRIPTION OF T HE PREFERRED EMBODIMENT 

15 

[001 8] In the following detailed description of the present invention, numerous specific details are set forth in order 
to provide a thorough' understanding of the present invention. However, it will be obvious. to one sidlled in the art that the 
present invention may be practiced without these specific details. In other instances well known methods, procedures, 
components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present 
20 Invention. 

[0019] The present invention Includes a method, apparatus and system Ibr standard voice user interface and voice 
controlled devices. Briefly, a standard voice user interface is provided to control various devices by. using standard 
speech commands. The istandard VUl provides a set of core VUl commands and syntax for the interface betwaeri a 
user and the voice controlled device. The core VUl commands include an identiflisation phrase to determine If voice con- 
25 trolled devices are available In an environment. Other core VUl commands provide for determining the names of the 
votoe controlled devices and altering them. 

[0020] Voice controlled devices are disclosed. A voice controlled device is defined herein as any device that is con- 
trolled by speech, which Is either audible or non-audible. Audible and non-audible are defined herein later A voice con- 
trolled device may also be refen-ed to herein as an appliance, a machine, a voice controlled appliance, a voice controlled 
30 electronic device, a nsime activated electronic device, a.speeoh controlled devtee, a voice activated electronic appli- 
ance, a voice activated appliance, a voice controlled electronic devtee, or a self-identifying voice controlled electronic 
device. . 

[0021] The present friventlon is controlled by and communicates using audible and non-audible speech. Speech as 
defined herein for the presfent Invention encompasses a) a signal or infprrtiation, such ttiat if the signal or Information 

35 were passed through a suitable device to convert it to variations in air pressure, the signal or Information could be heard 
by a human being and would be considered language, and b) a signal or Infonnatlon comprising actual variations In air 
pressure, such, that if a human being were to hear the signal, the human wouid consider it language. Audible speech 
refers to speech that a human can hear unassisted. Non-audible speech refers to any encodings or representations of 
speech that are not included under the definition of audible speech, including that whfch may be communicated outside 

40 the hearing range of humans and transmission media other than air. the definition of speech includes speech that is 
emitted from a human and emitted from a machine (including machine speech synthesis, playbacic of previously 
recorded human speech such as prompts, or other forms). 

[0022] Prompts which are communicated by a voice controlied device and phrases which are communicated by a 
user may be in languages or dialects other than English or a combination of multiple languages. A phrase is defined 
45 herein as a single word, or a group of words treated as a unit A user, as defined herelii, is a humaii or a device, includ- 
ing a votee activated device. Hence "a user's spoken phrase", 'a uiser Issuing a command", and ail other actions by a 
user Include actions by a device and by a human. 

[0023] Voice controlied devices include some type of speech recognition in order to be controlled by speech. 
Speech recognition and voice recognition are used synonomously herein and have the same meaning. Preferably, 

so speeker Independent speech recognition systems are used to provide the speech recognition capabllty of the voice 
controlled devices. Speaker independent speech recognitions systems are responsive to speaker-independent repre- 
sentations of speech. In the prefered embodiment, a speaker-Independent representafion of speech is a phonetic rep- 
resentation of speech. However, other speaker-independent representations of speech may also be used in 
accordance with the present invention. 

55 [0024] In order to gain access to the full, functionality of a voice controlled dwfce with the present invention, a user 
must communicate to the voice controlled device one of its associated appliance names, the appliance name may 
include one or more default names or one or more user-assignable names. A voice controlled device msy have a plu- 
rality of user^signable names associated with it in order to'provide personalized functionality to each user. 
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lap Irr time], th^ both must back off for a new randomly selected silence delay, but this time the delay must be of up to 
twice the length of the previous silence delay, but not to exceed 1 6 seconds. 

[0029] In order to restrict which voice controlled devices respond to an identification phrase, a user may include a 
voice controlled device's name in the identification phrase. For example, one could say "Socrates are you out there?" 
5 to see if a voice controlled device named Socrates was neartiy. Similarly, one could say "Clock are you out there" which 
would cause all voice controlled devices with an appliance name of Clock (whether a default appliance name or a user 
appliance name) to respond. A possible variation is that votee controlled devices may respond with some response 
other than thelrnames, as for example, might be needed for security reasons. 

[0030] A voice controlled device may use both visual and acoustfc identification methods. For example, even 
10 though a speech recognition engine is continuously on, it may still display the visual logo and / or other visual identifier. 
Similarly, in a voice controlled device that requires manual activation of the speech engine, once enabled, the engine 
could then be responsive to the command "What is out there?" 

[0031] In another aspect of the present invention, the initial storage of a user's spoken phrase (for example, when 
making a new phonebook entry under voice control) is processed by the speaker-independent speech recognition 
15 engine of the votee controlled devices. This engine returns a speaker-independent phonetic representation of the 
phrase. This speaker-independent phonetic representation is what is stored. 

[0032] When a command is issued by a user, it is also processed by the speaker-independent speech recognition 
engine of the present Invention. This could be the same speaker-Independent engine use for storing the original entries, 
or a completely different speaker-Independent engine. In either case, the engine returns a speaker-independent pho- 
20 netic representation of the command sequence. This speaker-independent phonetk: representation can be compared 
to earlier stored phonetic representations to determine whether the command Is recognizable. 
[0033] By converting both the stored spoken entries and any commands to speaker-Independent phonetic repre- 
sentation a number of advantages are provided. 

25 • Recognition will be reliable even if the user's voice has changed, perhaps due to a sickness, stress, fatigue, trans- 
mission over a noisy 01* distorting phone link, or other factors that might change a human user's or machine user's 
speech. Text-based information can be stored and then recognized. 

• Recognition will be reliable even if some other user had stored the original voice phrase, 

• Recognifion can be speaker-independent, even for user-stored commands and phrases. 

30 • Stored entries originating from text sources and from different speakers can all be combined and reliably for recog- 
nition. 

• The use of speaker-independent phonetic representations facilitates upgrading to Improved recognition engines as 
they become available. Improved speech recognition engines can use existing stored Information without Impacting 
reliability or requiring re-storage, since all stored entries are held In phonetic fomfi. New Infonnatlon stored using 

35 the Improved speech recognition engines can be used on equipment with older recognition engines. Old and new 
generations of equipment can interoperate without prior coordination by using phonetic representations. This 
allows, for example, two PDAs to exchange voice-stored phonebook entries and provide reliable recognition to the 
new users of that infomiation. Rnally, there are no legacy restrictions to hold back or restrict future development of 
speaker-independent recognition engines as long as they can create phonetic representations, unlike wavefomi- 

40 storage based systems, which must always be able to perform exactly the same legacy waveform transformaflons. 

VOICE CONTROLLED DEVICES 

[0034] Referring now to FIG. 1 A, environment 1 00 is illustrated. Environment 1 00 may be any communication envl- 
45 ronment such as an office, a conference room, a hotel room, or any kxiatlon where voice controlled devices may be 
located. Within environment 100, there are a number of human users 101A-101H, represented by circles. Also within 
the environment 1 00, are voice continued devkies 1 02A-1 02H, represented by squares and rectangles, each operation- 
ally controlled by the standard voice user Interface (VUt) of the present Iirventlon. Voice controlled devices 1 02A-1 02E, 
represented by rectangles, are fixed within the environment 100. Vdtee controlled devices 102F-102H, represented by 
so squares, are mobile voice controlled devices that are associated with human users 1 01 F-1 01 H respectively. Vokie con- 
trolled devces 102A-102H may be existing or future devices. Voice controlled devices lb2A-102E may be commonly 
associated with a user's automobile, home, office, factory, hotel or other locations where human users may be found. 
Altematively, if the voice controlled devices 102A-102E are to be controlled by non-audible speech, voice controlled 
devices may be located anywhere. 
55 [0035] In the present Invention, the standard VUI allows a user to associate a user-assignable name with these 
voice controlled devices 1 02A-1 02H. The user-assignable name of the votee controlled device may be generic such as 
telephone, clock, or fight Altematively, the name may be personalised such as those ordinarily given to humans such 
as John, Jim, or George. In either case, the votee controlled devices 102A'102H while constantly listenlrig will not 



■7 



EP 1 054 390 A2 



to SRS 204 and SRS 204 is coupled to APCC 206 . In the voice controlled device 1021, ACI 202 Is Its primary means 
of speech communication. 

[0039] Voice controlled device 1 02J includes ACI 202, SRS 204, APCC 206, communications Interface (ECl) 207, 
and connection 208. ACI 202 is coupled to SRS 204. APCC 206 is coupled to SRS 204. ECl 207 couples to SRS 204 
5 and connection 208 couples to tlie ECl 207. Voice controlled device 1 02J can alternatively communicate using speecli 
or voice communication signals througli ACI 202 or ECl 207. Voice controlled device 102K includes ACI 202, SRS 204, 
APCC 206, and an antenna 209. 

[0040] Voice controlled device 102K can communicate using audible speecli signals through the ACI 202 or using 
encoded speech signals through the ECl 207. ECl 207 couples to APCC 206. ECl 207 also couples to Connection 212. 
70 Connection 212 could, for example, be an antenna or Infrared port,. Voice controlled device 1 02L also Includes an ACI 
202, SRS 204, APCC 208, and an antenna 209. ACI 202 couples to SRS 204. SRS 204 couples to APCC 206. Antenna 
209 couples to APCC 206. Voice controlled device 1 02L can communicate by means of ACI 202 and APCC 206 through 
antenna 209. 

[0041 ] Voice controlled device 1 02M includes an APCC 206, SRS 204, an ECl 207, and connection 21 0. Connec- 
ts tlon 21 0 may be a wired or wireless connection, including an antenna. SRS 204 couples to APCC 206 and also to ECl 
207. Connection 21 0 couples to ECl 207. Voice controlled device 1 02M can communicate via ECl 207 over connection 
210. 

[0042] The APCC 206 represents the elements of the voice controlled device 102 that are to be controlled. For 
example, in the case of white goods, the items to be controlled may be temperature, a time setting, a power setting, or 

20 a cycle depending on the application. In the case of consumer electronics, the APCC 206 may consist of those items 
normally associated with buttons, switches, or knobs. In the case of telephone products, the APCC 206 may represent 
the buttons, the dials, the display devices, and the circuitry or radio equipment for making wired or wireless calls. In the 
case of automobile systems, the APCC 206 may represent Instrumentation panels, temperature knobs, navigational 
systems, the automobile radios channels, volume, and frequency characteristics. 

25 [0043] Refen-ing now to FIQ. 3, the voice controlled device 102 Is Illustrated. Voice controlled device 1 02, Illustrated 
in FIG. 3, is exemplary of the functional blocks within voice controlled devices described herein. Voice controlled device 
1 02 includes. the ACI 202, the APCC 206 and the SRS 204. The voice controlled device 1 02 may also have an ECl 207 
such as ECl 207A or ECl 207B. 

[0044] The ACI 202 illustrated in FIG. 3 Includes microphone 303. speaker 304, and amplifiers 305. The SRS 204 
30 as Illustrated in FIG. 3 includes the voice communication chip 301, coder/decoder (CODEC) 306 and 308, host micro- 
controller 310, power supply 314, power on reset circuit 316, quartz crystal oscillator circuit 317, mernory 318, and 
memory 328. The SRS 204 may optionally Include an AC power supply connection 315, an optional keypad 31 1 or an 
optional display 312. For bidirectional communication of audible speech, such as for local commandsi prompts and 
data, the speech communication path Is through the VCC 301, CODEC 306, and the ACI 202. For bidirectional com- 
as munication of non-audible speech, such as for remote commands, prompts and data, the non-audible speech commu- 
nication path Is through the VCC 301 , CODEC 308, ECl 207A or the VCC 301 , host microcontroller 31 0, APCC 206, 
and ECl 207B. The ECl 207 may provide for a wired or wireless.link such as through a telephone network, computer 
network, Internet, radio frequency flnk, or Infrared link. 

[0045] Voice communtealion chip 301 provides the votee controlled device 1 02 with a capability of communication 
40 via speech u^ng the standard votee user interlace of the present Invention. Mterophone 303 provides the voice control- 
led device 1 02 with the capability of listening for audible speech, such as voice commands and the device's appliance 
names. Microphone 303 may be a near field or far field microphone depending upon the application. For example, near 
field microphones may be preferable In portable ceil phones where a user's mouth Is dose while far field microphones 
may be preferable in carcell phones where a user's mouth is a distance away. Speaker 303 allows the voice controlled 
45 devtee 1 02 to respond using speech such as for acknowledging receipt of Its name or commands. Amplifiers 305 pro- 
vides amplification for the voice or speech signals received by the microphone 303. Additionally, the ampitflers 305 
allow amplification of representations of voice signal from the CODEC 306 out through the speakers 303 such that a 
human user 101 can properly Interfece to the voice controlled devfce 102. 

[0046] Microphone 303 and Speaker 304 are each transducers for converting between audible speech and repre- 
50 sentations of speech. CODEC 306 encodes representations of speech from the ACI 202 into an encoded speech signal 
for VCC 301 . In addition, CODEC 306 decodes an encoded speech signal from the VCC 301 into an representation of 
speech for audible communteation through the ACI 202. 

[0047] Alternatively, non-audible speech signals may be bi-dlrectlonally communicated by the voice controlled 
device 1 02. In this case, VCC 301 provides encoded speech signals to CODEC 308 for decoding. CODEC 308 decodes 
55 the encoder speech signal and provides it to the ECl 207A for communteaBon over the connection 1 05. Speech signals 
may be received overthe connection 1 05 and provided to the ECl 207A. The ECl 207A couples the speech signals Into 
the CODEC 308 for encoding. CODEC 308 encodes the speech signals Into encoded speech signals^ whtoh are cou- 
pled into the VCC 301 . 
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cycle. The MICROWIRE Interface 428 allows serial communication with the host microcontroller 310. The Ivlaster 
MICROWIRE controller 430 allows Interface to serial flash memory and other peripherals. The reset and configuration 
blocl< 432 controls definition of the environment of the voice communication chip 301 during reset and handles software 
controlled configurations. Some of the functions within the voice communication chip 301 are mutually exclusive. Selec- 

5 tlon among the alternatives is made upon reset or via a.Module Configuration register. The clock generator 434 Inter- 
faces to the quartz crystal oscillator circuit 31 7 to provide clodo for the various blocks of the voice communication chip 
including a real-time timer. The clock generator can also be used to reduce power consumption by setting the voice 
communksation chip 301 into a powerdown mode and returning it into normal operation mode when necessary. When 
the votea communication chip 301 Is in power-down mode, some of its functions are disabled and contents of some reg- 

10 isters are altered. The watchdog timer 436 generates a non-maskable lnten\ipt whenever software loses control of the 
processing units 402 and at the expiration of a time period when the voice communteation chip 301 is In a power-down 
mode. 

STANDARD VOICE USER INTERFACE 

15 

[00581 Similar to computer operating systems pr^>vlding a GUI, the standard voce user Interface (VUl) can be 
thought as being provided by a standard VUl operating system code. The standard VUl operating across a wide an-ay 
of voice controlled devices allows a user to interface any one of the voice controlled devices including those a user has 
never previously interacted with. Once a user is familiar with the standard VUl, they can walk up to and Immediately 

20 start using any vokse controlled device operating with the standard VUl. The standard VUl operating system code has 
specific standardized commands and procedures in which to operate a voice controlled devfce.. These standardized 
commands and procedures are universal to machines executing the standard VUl operating system code. Voice con- 
trolled applteation software. operatlng with the standard VUl operating system code, can be written to customize voice 
controlled devfces to speciflc applteatlons. The voice controlled applteation software has voice commands specific to 

25 the applteation to which the votee coritrolled devtee Is used. A particular votee controlled device may also have addi- 
tional special features that extend ttie core capabilities of the standard VUl. 

[0059] Some of the standard VUl functionality in the core VUl include a way to discover the presence of voice con- 
trolled devices, a core common set of commands for all voice controlled devices, a way to learn what commands (both 
core commands and appliance-specific commands) the voice controlled device will respond to, a vocalized help system 
30 to assist a user without the use of a manual or display, a way to personalize the voice controlled device to a user with 
user assignable settings, security mechanisms to control use of voice controllied devtees to authorized users and pro- 
tect user assignable settings and Information from other users, and standard w^s for a user to interact with voice con- 
trolled devices for common operations (e.g. selecting yes or no, listing and selecting Items from a list of options, 
handling en-ors gracefully, etc.). The standard VUl Includes an API (Applications Programming Interface) to allow Sott- 
as ware developers to write custom votee controlled applications that Interface and operate with the standard VUl and 
extend the voice controlled command set. 

[0060] Refen-ing now to FIG. 5, a block diagram Illustrates the Sofhware 500 for controlling Votee Controlled Device 
1 02 and whteh provides the standard VUl and other functfonality. The Software 500 Includes Applteation Code 61 0, a 
VUl software module 512 and a Vocabulary 524. Application code 510 may be further modified to support more than 
40 one applteation, representing multiple application code modules, to provide for further customizalton of a voice control- 
led devfce 1 02. The Vocabulary 524 contains the phrases to be detected. The phrases within the Vocabulary are divided 
into groups called Topics, of which there may be one or more. In Rgure 5, the Vocabulary 524 consists of two Topics, 
Topic 551 and Topic 552. 

[0061] Typically, Application Code 510 Interfaces to the VUl software 512 through the Application Programming 
45 Interface (API) 507. The VUl software 512 provides special services to the Application Code 510 related to votee inter- 
face, including recognition and prompting. The inten-eiationship between the VUl software 51 2 and the application code 
51 0 is analogous to that between Microsoft's MS Windows and Microsoft Word. Microsoft Windows provides special 
services to Microsoft Word related to displaying Items on a screen and receiving mouse and keyboard inputs. 
[00621 Generally, the Applicaflon Code 51 0 may be stored In host memory and executed by the host microcontroller 
50 31 0. However, the functionality of the host microcontroller 310 can be embedded Into the VCC 301 such that only one 
devtee or processor and one memory or stdrage device is needed to execute the code associated with the software 500. 
[0063] All phrases that can be recognized, Including those phrases for the core arid applteation specif te commands, 
are included In the Vocabulary 524. The VUl software module 512 can directly access the vocabulary phrases, for 
example for use during recognition, the VUl software module 512 can also processes tokens. Tokens abstractly relate 
55 to the phrases within the Toptes 551 -552. Tokens are Integer numbers. For example, the phrase for 'dial' might have a 
token value ot 'S', and the phrase for 'hang-up' might have a token value of '6*. there Is a token value assigned to every 
phrase that can be recognized. Because the VUl software module 512 can process tokens related to the vocabulary file 
524, it can refer to phrases without having to directly access them. This makes It possible to change languages (from 
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ground speech may still be present The (name) is the appjlance name associated with a voice controlled device 1 02. 
The {command) Is an operation that a user wants performed. The (modifiers & variables) conslstof additional information 
needed by some commands. The SRS 204 recognizes the elements in their syntax in order for a user to control voice 
controlled devices. 

5 [0072] Most voice controlled devices will continuously listen forthe voice command sequence. When a voice con- 
trolled device hears Its (name), It knows that the following (command) Is Inteaded for It. Since each user has a different 
(name) for a voice controlled device, the (name) also ujilquely identifies the user, allowing the voice controlled device to 
select that user's personalization settings. Commands include core VUl commands included with ail voice controlled 
devices, and commands specific to a given application, all of which are stoted within the vocabulary 524. 

w [0073] Requiring (silence) before detection of (name) helps prevent false detection of (name) during norniai conver- 
sational speech (I.e. during periods when the user is speaking conversationally to other users and not to the voice con- 
trolled device). In all cases, the duration of (silence) can be configured by the manufacturer and can range from 0 (no 
(silence) required) to a second or more. Typically it will be about a quarter of a second. 

[0074] Examples of voice comniand sequences that might be used With a voice controlled device such as a tele- 
15 phone named Aardvark Include "Aardvark Call The Office", "Aardvark Dial 1-800-55-1212", and "Aardvark Hang-up". 
(In the command examples and descriptions provided, for the sake of brevity the (silence) is often not shown, and even 
where it is shown or described, the opHon aiways exists of a manufacturer choosing to use a silence duration of zero.) 
[0075] There are two special cases where the command syntax Is pemnitted to differ from the general syntax. The 
first spedal'case is in voice contrblied devices that do not contlnuousiy listen for (silehceXname). For example, in some 
20 battery ojaerated applications, power consumption iimltaUons may require the VCC 301 in the voice controlled device 
1 02 to be powered down during idle periods. Another example is a voice controlled device located wfiere false recog- 
nition of a name would have undesirable results, for example, a desktop plione in a conference room during a presen- 
tation. A third example is voice controlled devtees where there Is a high risk of false recognition, for example, where 
multiple conversations can be heard. 
2s , [0076] l^r these types of situations, an alternate command syntax is used in conjunction with a button or switch of 
soma type. The first alternate command. syntax is: 

(acHvatlon of a sw*«*) (silence (optionBi)><name> (command) (modifiers & variables). 

30 In this syntax, the (actlvafion of aswitch) means the user presses a button or performs some other mechanical act (e.g. 
opening a flip-style cell phone) to activate the recognition capability. 

[0077] A second special case Is where the user nomiaily enters a series of commands in quick succession. For 
these cases, the user can Identify themselves once to the voice controlled devtoe using a password protection method, 
or by Issuing a command that Includes the voice controlled devtee's appliances (name), and thereafter continue entering 
35 commands. The second alternate command syntax (in this example, for three successive commands) is: 

(silence) <nameV <c6mmand><modlfiers & variables needed! 
(silence) (name (opUbnal)) (command) (modifiers & variables as needed) 
(silence) (name (optional)) (command) (modifiers & variables as needed). 

40 

With this syntax, the user can issue a series of corrimands without having to constantly repeat the voice controlled 
device's appliances (name). However, the user is permitted to say the (name) at the start of a command. Note that In this 
syntax, the (silence) is required to properiy recognize the spoken (name) or (command). 

[0078] When either of the first or second alternate syntaxes is used, it is desirable to ensure that If a new user starts 
45 woridng with the voice controlled devk;e, they are properiy Identified. Tiiis can be ensured by explicitly requiring the 
(nama> after a period of Inactivity or after power-up of the voce contnolied device or other similar protocol. 

STANDARD CORE VUl COMMANDS 

so [0079] There are a number of standard core commands included in the vocabulary 524 of voice controlled devices 
102 operating using the standard VUl. FIGs. 6A-8 illustrate the syntax of ttie following commands. 
[0080] Referring to FIG. 6A, at start 600, the appliance name, (name), of a voice controlled device is usually spoken 
prior to a command. Any of the voice controlled device's appliances names can be spoken whenever the voice control- 
led device is listening for a comrhand. If the (name) Is not followed by a command within some period of time, the voice 

55 controlled deytea Will go back to return to start 600 in its original idle state. This Is Indicated by the solid box Silence of 
N seconds. N in this case is a programmable value usually application dependent.and assigned by the voice controlled 
device manufecturer. After supplying the appliance name, a user is granted access to further commands of the standard 
VUl operating on the votee controlled device at 601 . 
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4 to (for example) Doggone. If the user attempted to change a fifth user-assignable name in sequence with the com- 
mand (Telephone change your name "), It would result in an error message because all available user-assignable 
appliance names were assigned. Note that the voice controlled device always responds to the fectory programmed 
name, even if aii user-assigned names are defined. Accordingly, in this example of a fifth attempt, the voice controlled 
device still recognizes the Telephone" factory programmed name - it is just unable to assign afifth new user-assignable 
appliance name. 

[0088] An existing user-assignable appliance name can also be changed with the "Change Your Name " command. 
Continuing the above example, "Aardvarit change your name " would alter the appliance's name for the first user (for 
example, it could be changed to Platypus), and leave the other three user names unchanged. Similarly, 'Platypus 
change your name * followed by a dialog to set the name to Telephone" would reset the first user name to the factory- 
programmed default. 

Identification of Voice Controlled Devices 

[0087] As voice controlled devices proliferate. It is Important that users be capable of readily Identifying what, if any, 
voice controlled devices are present when they enter a new environment. For example, a user wall« into a hotel room 
that has a number of devices. In order to use them a user needs to know which devtees are voice controlled devices. 
Additionally a user needs to know the appliance names in order to properly control them. Beside being audibly identi- 
fied, voice controlled devices can be Identified visually as well as Isy using a logo signifying a voice controlled device 
utilizing the standard VUl. 

[0088] Acoustic identification works when voice controlled devices are actively listening for recognizable com- 
mands. In most cases, this means the voice controlled device Is constantly listening and attempting recognition. Typi- 
cally, these voice controlled device® will be AC powered, since the power drain from continuous recognition will be 
unacceptable for most battery operated voice controlled, devices. Referring to FIG. 6A and 6C, the acoustic identifica- 
tion is accomplished by a user communicating an identification phrase to command the voice controlled device. The 
identification phrase "What Is Out There?" or some other suitable identification phrase may be used for causing the 
voice controlled devices to identify themselves. 
[0089] The syntax of the standard VUl Identification phrase Is: 

(silence) What Is Out There? 

in response to this query, any. voice controlled device that hears the question must respond, the typical voice controlled 
devices response Is a rendom delay of up to 2 seconds of relative silence, followed by a beep (the standard signal) , 
and the response 'You can call me (name>*, where (name) Is the factory-programmed name that can be used to address 
the voice controlled devk:e. In the telephony voice controlled device example described above, a response rrilght be 
'(beep) You can call me Telephone.' 

[0090] Referring to FIG. 6C, during the random delay of up to 2 seconds, each responding voice controlled device 
listens for another votes controlled device's response (specifically, for another voice controlled device's beep). In the 
event another votee controlled device starts responding (as evidenced by a beep) during this silence period, the listen- 
ing voice controlled device nfiust restart Its silence timing after the responding voice controlled device finishes. In the 
event two vofce controlled devices start responding at the same time (overlapping beeps), they both must back off for 
a new randomly selected silence delay. However, this time the random delay may be greater than the first; up to twice 
the length of the previous silence delay. In any event, the delay should not exceed 1 6 seconds. Additional back off peri- 
ods for further confltet resolution is provided (f other voice controlled devices respond. 
[0091] Refen'ing to FIG. 6A, the syntax of the Request User-Assignable Names command is: 

(name) Tell Me Your Name 
or 

(name) Tell Me Your Names 

If security permits, any user-programmed <name) or the defeult (name) can be used. The Request User-Assignable 
Names command is used to asik a voice controlled device to list all the user-programmed (names) that It will respond to. 
If security perniHs, the voice controlled device communicates each use-programmed name in a list fashion, Between 
each user-assigned name It pauses for a moment During this pause a user may communk:ate a command to the voice 
controlled device and It will be executed as if given with that user-programmed (name). For example consider the teleph- 
ony voice controlled device example above. The command "Telephone Tell Me Your Name' provided after a pause wiil 
cause the telephone to respond iiy s^ing °l have been named Aardvark, (pause) Barracuda (pause). Coyote (pause), 
and Doggone (pause).* During the pause that followed the voice controlled device saying'Coyote", a user may say 'Call 
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[0102] The syntax of the Call command is: 

<name) Call <voicetag) 
or 

5 <name) Call<cllglts> 

The Call command is used to dial a specific phone number, ecpressed either as a series of digits or as a phonebook 
voicetag..The <digits> can be any list of numeric digits. The telephony voice controlled device allows for the synonyms 
"oh" for zero, and "hundred" for zero-zero to be enabled. The sequence of (digits) can contain embedded pauses. How- 

10 ever, if a pause exceeds a programmable duration, the sequence is temtiinated and the command executed after rec- 
ognition of a pause that exceeds a duration set by the system designer. The telephony voice controlled device response 
to a Call command should be "Calling (digits)" or "Calling (voicetag)" with the recognized digits or recognized voicetag 
voiced to verify accurate recognition. The "Cancel" command can be used to cancel the calling operation in the event 
of misrecognitlon. 

15 [0103] The syntax of the Dial command Is: 

(name) Dial (voicetag) 
or 

(name) Dial (digits) 

20 

The Diial command is the same as the Call command. 
[0104] The syntax of the Answer command is: 

(name) Answer 

2S 

This command is used to answer an incoming call. The response prompt is "Go ahead". 
[0105] The syntax of the Hangup command is: 

(name) IHangup 

30 

This command Is used to hangup an acflve call. The response prompt is a high-pitched beep. 
[010^ The syntax of the Redlal command Is: 

(name) Redlal 

35 

This command is used to redlal a number. The response is "Redialing (digits)" or "Redialing (voicetag)", depending on 
whether the previous Call or Dial command was to (digits) or a (voicetag). If there was no eariier call made, the response 
is "Nothing to redlar: 

[0107] The syntax of the Store command Is: 

40 

(name) Store 

The Store command Is In the phonebook submenu and is used to add a new voicetag. 
[0108] The syntax of the Delete command is: 

4S 

(name) Delete 

The Delete command is In the phonebook submenu and is used to delete a voicetag. 
[0109] The syntax of the Mute command is: 

50 

(name) Mute 

This command mutes the microphone. The response by the voice controlled device is "Muted". 
[0110] The syntax of the Online command is: 

5S 

(nanne) Online 

. This command unmutes the microphone. The response Is "Online". 
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name" 


nuinber" 




5 


"Please repeat the new 


"The nvimber for 
<voicetag> is <digits>. 
IS this correct?' 


"nine" 


10 


" PXease Sciy tlie nuihl^eir 
for <voicetag>" 


"The nuinhex* fox* 
<voicetag> has been 




15 




stored* 






"Tliat name Is not in the 


"Do you want to store it 


"hundred" 




phone book* 


now? " 




20 




"Muted" 


"Nothing to 
redial" 


2S 






"Star' 








"Flash* 








"Pound* 



30 



[0112] In addition to these prompts, tlie voice controlled devices can generate a number of different tones or beeps. 
These Include a medium pitch beep (e.g. 200 millisecond, 500 Hz. sine wave), a low pitched beep (e.g. a buzzer sound 

39 or 250 millisecond, low frequency beep signifying enoneous entry) and a high pitched beep (e.g. 200 milliseconds, 
1 200 Hz. sine wave). Other sounds are possible and would be within the intended scope of the present Invention. 

Vocabulary For Telephone Answering Voice Controlled Device 

40 [0113] In addition to the forgoing, application specific commands for the standard VUl enable a user to interface to 
a telephone answering voice controlled device using voice commands. A user can manage message functions and 
obtain remote access from a telephone answering voice controlled device without using a keypad. The following lists 
the additional voice commands to be included In the vocabulary 224 for telephorie answering voice controlled device. 



<najne> Play new 


<name> Rewind <n> 


<name> Stop 


<nanie> Play all 


<name> Record Greeting 


<name> Play Greeting 


<name> Delete this 


<naine> Record message 


<name> Room monitor 


<nanie> Delete all 
messages 


<name> Answer On 


<name> Password 
<password phrase> 


<name> Forward <n> 


<naine> Answer 0£f 
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1 03. If the recognition of the response was not successful, the anay Is two elements long. The first element Is set to 
zero and the second element Indicates the type of error that occuned. In this case, Element 1 is set to 0 indicating that 
an error was detected. Element 2 Is set to 1 7 indicating that a response was not detected In the allowed time (Timeout 
error) or 18 Indicating that a response was detected, but it was not recognizable (out-of-vocabulary-word en-or). The 
5 array returned for a timeout error is two elements long with values 0, 1 7 and the anay returned for an out-of-vocabulary- 
word error is two elements long with values 0, 18. 

[0119] Refefflng to FIG. 1 1 , GETRESPONSEPLUS user Interface function plays a Prompt to a user that solicits a 
response and waits for the response. GETRESPONSEPLUS Is similar to GETRESPONSE In that it plays a Prompt for 
the user and then waits for a spoken response. However. G ETRESPONSEPLUS indudes the capability to play prompts 

10 to recover from error situations where the user has not spoken or has excessive noise in the background. GETRE- 
SPONSEPLUS listens for a spoken response that matches the topics in TopicLlst. GETRESPONSEPLUS either 
returns an anay of recognized tokens, or an error indicator. The parameters for GETRESPONSEPLUS are 
lnitlal_Prompt, Timeout, STS_Sound. TopicLlst, MaxTries, lntervene_Prompt, Repeat.Prompt, and the Help_Prompt. 
The InltlaLPrompt parameter Is the initial prompt to be played to a user to solicit a response. The TimeOut parameter 

IS Is the number of milliseconds to wait for a response before flagging that a response was not detected. The STS_Sound 

. prompt is a sound or prompt to be played if user speaks before Prompt finishes playing. Typically. STS_Sound prompt 
will be a short tone or beep sound rather than a spoken phraise. The parameter TopicLlst Is the vocabuiary subset for 
the list of topics which this SRS 204 should use to Identify the spoken response. The MaxTries parameter is the maxi- 
mum number of times GETRESPONSEPLUS will re-prompt the user in an effort to get a good recognition. If recognWon 

20 does not occur after MaxTries, GETRESPONSBI=LUS will return and Indicate an error. The InterveneiPrompt param- 
eter Is a prompt played to ask the user to repeat himself (e.g. "There was too much noise. Please repeat what you 
said."). This prompt Is played when there was too much noise during the previous recognition attempt. The 
Repeat_Prompt parameter Is the prompt played to ask the user to repeat what was jiist said (e.g. "Please repeat what 
you said"). This prompt is used when a spoke-too-soon en^r occurred. The Help_Prompt parameter Is the prompt 

25 played when the user seems to need further instructions, including when the user says nothing. The voice controlled 
device returns a pointer to an integer array upon completion of the user Interface function. If the recognition of a 
response associated with the TopicLlst was successful, the first element in the array Is the number of tokens returned 
and the following elements in the array are the tokens for each Identified speech element (one or more words). Element 
1 is n the Number of tokens returned. Elements 2 through n+1 are the Token values for each speech element recog- 

30 nized. For example, consider the phrase "Telephone Dial Office". If the token value for the speech element "Telephone" 
is 7, for the speech elerhent "Dial" Is 12. and forthe speech element "Offloe" is i03, then If they are all recognized suc- 
cessfully, the complete arr^ returned would be four eiemerits long with the values 3, 7, 1 2, 1 03. If recognition vras. not 
successful, the anay is four elements long. The first element Is zero. The second element indicates the most recent type 
of error that occurred. The third through fifth elements indicate the number of times each type of error occurred between 

35 when GETRESPONSEPLUS was called to when GETRESPONSEPLUS retumed. In this case Element 1 has a value 
of 0 indicating that an en-or was detected. Element 2 has a value of 17 Indicating that a response was not detected in 
the allowed time (TimeOut enor) or 18 Indicating that a response was detected, but it was not recognizable (out-of- 
vocabulary-word en-or) or 19 Indicating that a spoke-to-soon enorwas detected. Element 3 has a value of x Indicating 
the number of times a TimeOut en-or was detected. Element 4 has. a value of y.indlcaflng the number of times an out- 

40 of-vocabulary-wotd error was detected. Element 6 has a value of z Indlcafing the numberof times a spoke-toO-soon 
error was detected, 

[0120] Refening to FIG. 12, LISTANDSELECT user interface function first plays a Prompt Then it plays each 
prompt in anay LIstOfMenuPrompts; pausing after each for a PauseTlme. During these pauses, the recognizer listens 
for a spoken response that matches the topics In TopicLlst LISTANDSELECT either returns an array of recognized 

45 tokens, or an enxsr Indicator. The parameters for LISTANDSELECT include Inltlal.Prompt, Timeout STS_Sound, Top- 
icLlst, LIstOfMenuPrompts, PauseTlme, and the Help_Prompt. The InltlaLPrompt parameter Is the Initial prompt to be 
played to the user The TimeOut parameter Is the number of milliseconds to wait for a response, after playing all the 
prompts in ListOfMenuPrompts, or before flagging that a response was not detected. The STS_Sound parameter is the 
sound orprompt tb be pleyed If user speaks before aprompt finishes playing. Typically, STS_Sound will be a short tone 

so orteep sound rather than a spoken.phrase. The parameter TopteLfet is the vocabulary subset for the list of topics which 
the SRS 204 should use to identify the spoken response. The UstOfMenuPrompts parameter is an array of prompts 
which will be played one at a time. The first element In the anay is a count of the number of prompts In LlstOfMenu- 
. Prompts. The PauseTlme parameter Is the time to pause after playing each prompt in UstOfMenuPrompts. The Pause- 
Time parameter has a value in mllllseoonds/ The He|p_Prompt parameter is the prompt played when the user seems to 

ss need further insfrucBons, Including when the user s^ nothing. The voice controlled devkse returns a pointer to an Inte- 
ger array upon completion of the user Interface function. If recognition was successful, the first element in the airay is 
the number of tokens returned, and the following elements in the anay are the tokens for each Identified speech ele- 
hieht (one or..more words). Element 1 has a value of h Indicating the number of tokens returned. Elements 2 through 
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Machine Requests to Humans 

[0125] Machines can ask humans to do things. Any request should be polite. For example, a voice activated cellular 
telephone might asl< to be placed in its charger when its batteries are running low. Humans should always have the 
5 option to refuse a machine's request, and the machine should politely accept that, unless the machine considers the 
situation threatening to human life or valuable data, in which case its protests can be more urgent. 

Machines That Use the Telephone On Their Own 

10 [0126] If a voice controlled device answers the telephone, or places a call to a human user, It should clearly Identify 
itself as a machine if there is any risk of it being considered human. 

Recording User Speech 

IS [0127] No machine should record or transcribe a human user's conversations unless those humans present are 
aware that this is occuning. 

Volume Levels 

so [01 28] Machines should modulate their volume levels in response to ambient noise levels, unless specifically over- 
ridden by a human. Machines should be sensitive to when humans want them to be silent (for example, when humans 
are sleeping). Machines shouldn't babbie needlessly, and should permit a user barge-in as a means to silence them. 

Machine- to-Maohlne Communication 

25 

[0129] FIG. 13 Is a block diagram of a pair of voice controlled devices 102M and 102N (each also referred to as a 
machine) communicating, neither, one or both of whtoh could be using the standard voice user interface 500 of the 
present invention in the communication environment 1 300. Voice controlled devices can talk to each other to find out 
what other voice controlled devices are present, what kinds of information they understand, and to exchange informa- 

30 tion. For example, a voice controlied .TV may ask a voice controlled VCR about necessary settings for it to operate, 
Machlne-to-maohine communteation between voice controlled devices occurs In both audible and non-audible formats. 
Essentially, machine-to-machine communication using speech may occur ovisr any speech-compatible media, including 
sound w^s through air, conventional telephone links, internet votee links, radio votoe channels, and the like. Machlne- 
to-machlne communication caii occur where none of the machines, some of the machines, or ail of the machines 

35 include the VUlof the present invention. 

[0130] Using the standard VUl, a voice controlled device can locate other voice controlled devtees within a commu- 
nications environment in a number of ways. These Include overhearing a human interact with another machine, over- 
hearing a machine Interact with another machine, explicitly requesting neartiy machines to identify themselves by using 
the identjfkiation phrase '(silence) What is out there?*, explicitly seeking a specific class of machines (e.g. all clocks) by 

40 addressing them by a name category "(silence) Clock are you out there?*i or explfclfly seeking a specific machine (e.g. 
a dock named Socrates) by addressing it by name "(Silence) Socrates are you out there?". 

[0131] In the first two cases, the process of listening to other conversations would reveal the other machines' 
names. In the otherthree cases the machines within earshot who respond to thfe "are you out there' command would 
respond with their names. In the last two cases, the 'What is out there?" command is restricted to certain classes of 
45 machines and a spediFic named machine thereby limiting the number of machines that will respond to the command. 
Once the name of the target voice controlled device is known, the initiating voice controlled device can Issue other com- 
mands (e.g. "Socrates what time is it?") to the other. 

[0132] in some cases, a voice controlled device may need to talk to another voice controlled device, one or both of 
which may not adhere to the above protocol. In these cases, the machines can be explicitly programmed to Issue the 
50 con^ect commands and recognize appropriate responses. A simple example of this interaction would be a voice control- 
led device with voice recognition capability and a telephone voice interface dialing a voice-based service such as a spo- 
ken report of the time, and simply capturing the desired data (the time). 

[0133] The preferred embodiments of the present invention for METHOD AND APPARATUS FOR STANDARD 
VOICE USER INTERFACE AND VOICE CONTFIOLLED DEVICES are thus described. While the preferred embodi- 
55 ments of the present invention utilize a speaker independent voice recognition system, the present invention Is also 
compatable with speaker dependent vofce recognition systems. While the present invention has been described In par- 
tteular embodiments, the present invention should not be construed as limited by such embodiments, but rather con- 
strued according to the claims that follow below. 
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the communicated appliance name and the command are communicated using audible speech. 

10. The method of claim 8 for activating a voice controlled dwice, wherein, 

5 the communicated appliance name and the command are communicated using non-audible speech. 

11 . A method of controlling a voice controlled device, the method comprising: 

providing a voice controlled device having a speech recognition system for recognizing speech; 
10 storing a defeuit appliance name into the voice controlled device; 

communicating a communicated name and a command to the voice controlled device; and 
controlling the volce.oontrolled device if the communicated name is recognized as matching the default appli- 
ance name and the command Is recognized by the voice controlled device. 

15 12. The method of claim 1 1 for activating a voice controlled device, wherein, 

the communlcatsd appliance name and the command are communicated using audible speech. 

1 3. The method of claim 11 for activating a voice controlled device, wherein, 

20 

the communicated appliance name and the command are communicated using non-audible speech. 

14. A method for activating a voice controlled device, the method comprising: 

25 providing a voice controlled device having a speech recognition system for recognizing speech; 

storing a default appliance name Into the voice controlled device; 

storing at least one user assignable appliance name Into the voice controlled device; 

communicating a communicated name and a command to the voice controlled device; and 

controlling the voice controlled device If the communicated name is recognized as matching the at least one 
30 user assignable appliance name or the default appliance name and the command is recognized by the voice 

controlled device. 

1 5. A method of assigning a new name to a voice controlled device, the method comprising: 

35 providing a voice controlled device having a speech recognition system for recognizing speech; 

activating the voice controlled device; and 

communicating a new name to the voice controlled device at least once. 

1 6. The method of dalrh 1 5 for assigning a tiew name to a voice controlled device, wherein, 

40 

the voice controlled device is activated by communicating a current appliance name and a change name com- 
mand. , . 

1 7. The method of claim. 1 5 for assigning a new name to a voice controjled device, wherein, - 

45 

the new name is communicated, using audible speech. 

1 8. The method of claim 1 5 for assigning a new name to a voice controlled device, wherein, 
so the new name is communicated using non-audible speech. 

19. The method of claim 1 5 for assigning a new name to a voice controlled device, wherein: 

the voice controlled device includes prompting capability and the voice controlled device communicates audi- 
ss ble prompts to a user In order to request communication from the user of the new name. 

20. The method of dalm 15 for assigning a new name to a voice controlled device, wherieln: 
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a security means to protect each voice controlled device from unauthorized use. 

29. The fitst voice controlled device of claim 24 capable of operating in a communication environment with at least one 
other voice controlled device, the first voice controlled device further comprising: 

a security means to protect each voice controlled device from unauthorized use. 

30. The first voice controlled device of claim 27 capable of operating in a communication environment with at least one 
other voice controlled device, the first voice controlled device further comprising: 

a security means to protect each voice controlled device from unauthorized use. 
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